Papers with Inductive transfer learning
Universal Language Model Fine-tuning for Text Classification (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to computer vision require task-specific modifications and training from scratch. |
| Approach: | They propose a method that can be applied to any task in NLP and propose to open-source it. |
| Outcome: | The proposed method outperforms the state-of-the-art on six text classification tasks, reducing error by 18-24% on majority of datasets. |
BARThez: a Skilled Pretrained French Sequence-to-Sequence Model (2021.emnlp-main)
Copied to clipboard
| Challenge: | Inductive transfer learning has taken the entire NLU field by storm, with models such as BERT and BART setting new state-of-the-art on countless tasks. |
| Approach: | They introduce a large-scale pretrained seq2seq model for French that is very competitive with state-of-the-art BERT-based French language models such as CamemBERT and FlauBERT. |
| Outcome: | The proposed model outperforms existing models on discriminative and generative tasks on a French summarization dataset. |